Reduced area imaging devices incorporated within surgical instruments

ABSTRACT

A reduced area imaging device is provided for use in medical or dental instruments such as an endoscope. In one configuration of the imaging device, the image sensor is placed remote from the remaining circuitry. In another configuration, all of the circuitry to include the image sensor is placed in a stacked fashion at the same location. In a first embodiment of the invention, the entire imaging device can be placed at the distal tip of an endoscope. In a second embodiment, the image sensor is remote from the remaining circuitry according to the first configuration, and wherein a control box can be provided which communicates with the image sensor and is placed remotely from the endoscope. In another embodiment, the imaging device can be incorporated in the housing of a standard medical camera which is adapted for use with traditional rod lens endoscopes. In yet another embodiment, the imaging device can be wholly incorporated within the endoscope by placing the image sensor and timing and control circuitry within the distal tip of the endoscope, and placing the remaining processing circuitry within the handle of the endoscope. In any of the embodiments, the image sensor may be placed alone on a first circuit board, or timing and control circuits may be included on the first circuit board containing the image sensor. One or more video processing boards can be stacked in a longitudinal fashion with respect to the first board, or the video processing boards may be placed in the control box or in the handle of the endoscope.

This application is a continuation-in-part application of U.S. Ser. No.08/976,976, filed Nov. 24, 1997, and entitled “Reduced Area ImagingDevices Incorporated Within Surgical Instruments.”

TECHNICAL FIELD

This invention relates to solid state image sensors and associatedelectronics, and more particularly, to solid state image sensors whichare configured to be of a minimum size.

BACKGROUND ART

In recent years, endoscopic surgery has become the accepted standard forconducting many types of surgical procedures, both in the medical anddental arenas. The availability of imaging devices enabling a surgeon ordentist to view a particular surgical area through a small diameterendoscope which is introduced into small cavities or openings in thebody results in much less patient trauma as well as many otheradvantages.

In many hospitals, the rod lens endoscope is still used in endoscopicsurgery. The rod lens endoscope includes a very precise group of lensesin an elongate and rigid tube which are able to accurately transmit animage to a remote camera in line with the lens group. The rod lensendoscope, because of its cost of manufacture, failure rate, andrequirement to be housed within a rigid and straight housing, is beingincreasingly replaced by solid state imaging technology which enablesthe image sensor to be placed at the distal tip of the investigatingdevice. The three most common solid state image sensors include chargedcoupled devices (CCD), charge injection devices (CID) and photo diodearrays (PDA). In the mid-1980s, complementary metal oxide semiconductors(CMOS) were developed for industrial use. CMOS imaging devices offerimproved functionality and simplified system interfacing. Furthermore,many CMOS imagers can be manufactured at a fraction of the cost of othersolid state imaging technologies.

One particular advance in CMOS technology has been in the activepixel-type CMOS imagers which consist of randomly accessible pixels withan amplifier at each pixel site. One advantage of active pixel-typeimagers is that the amplifier placement results in lower noise levelsthan CCDs or other solid state imagers. Another major advantage is thatthese CMOS imagers can be mass produced on standard semiconductorproduction lines. One particularly notable advance in the area of CMOSimagers including active pixel-type arrays is the CMOS imager describedin U.S. Pat. No. 5,471,515 to Fossum, et al. This CMOS imager canincorporate a number of other different electronic controls that areusually found on multiple circuit boards of much larger size. Forexample, timing circuits, and special functions such as zoom andanti-jitter controls can be placed on the same circuit board containingthe CMOS pixel array without significantly increasing the overall sizeof the host circuit board. Furthermore, this particular CMOS imagerrequires 100 times less power than a CCD-type imager. In short, the CMOSimager disclosed in Fossum, et al. has enabled the development of a“camera on a chip.”

Passive pixel-type CMOS imagers have also been improved so that they toocan be used in an imaging device which qualifies as a “camera on achip.” In short, the major difference between passive and active CMOSpixel arrays is that a passive pixel-type imager does not perform signalamplification at each pixel site. One example of a manufacturer whichhas developed a passive pixel array with performance nearly equal toknown active pixel devices and being compatible with the read outcircuitry disclosed in the U.S. Pat. No. 5,471,515 is VLSI Vision, Ltd.,1190 Saratoga Avenue, Suite 180, San Jose, Calif. 95129. A furtherdescription of this passive pixel device may be found in co-pendingapplication, Ser. No. 08/976,976, entitled “Reduced Area Imaging DevicesIncorporated Within Surgical Instruments,” and is hereby incorporated byreference.

In addition to the active pixel-type CMOS imager which is disclosed inU.S. Pat. No. 5,471,515, there have been developments in the industryfor other solid state imagers which have resulted in the ability to havea “camera on a chip.” For example, Suni Microsystems, Inc. of MountainView, Calif., has developed a CCD/CMOS hybrid which combines the highquality image processing of CCDs with standard CMOS circuitryconstruction. In short, Suni Microsystems, Inc. has modified thestandard CMOS and CCD manufacturing processes to create a hybrid processproviding CCD components with their own substrate which is separate fromthe P well and N well substrates used by the CMOS components.Accordingly, the CCD and CMOS components of the hybrid may reside ondifferent regions of the same chip or wafer. Additionally, this hybridis able to run on a low power source (5 volts) which is normally notpossible on standard CCD imagers which require 10 to 30 volt powersupplies. A brief explanation of this CCD/CMOS hybrid can be found inthe article entitled “Startup Suni Bets on Integrated Process” found inElectronic News, Jan. 20, 1997 issue. This reference is herebyincorporated by reference for purposes of explaining this particulartype of imaging processor.

Another example of a recent development in solid state imaging is thedevelopment of CMOS imaging sensor which is able to achieve analog todigital conversion on each of the pixels within the pixel array. Thistype of improved CMOS imager includes transistors at every pixel toprovide digital instead of analog output that enable the delivery ofdecoders and sense amplifiers much like standard memory chips. With thisnew technology, it may, therefore, be possible to manufacture a truedigital “camera on a chip.” This CMOS imager has been developed by aStanford University joint project and is headed by Professor Abbasel-Gamal.

A second approach to creating a CMOS-based digital imaging deviceincludes the use of an over-sample converter at each pixel with a onebit comparator placed at the edge of the pixel array instead ofperforming all of the analog to digital functions on the pixel. This newdesign technology has been called MOSAD (multiplexed over sample analogto digital) conversion. The result of this new process is low powerusage, along with the capability to achieve enhanced dynamic range,possibly up to 20 bits. This process has been developed by AmainElectronics of Simi Valley, Calif. A brief description of both of theprocesses developed by Stanford University and Amain Electronics can befound in an article entitled “A/D Conversion Revolution for CMOSSensor?,” Sep. 1998 issue of Advanced Imaging. This reference is alsohereby incorporated by reference for purposes of explaining theseparticular types of imaging processors.

The above-mentioned developments in solid state imaging technology haveshown that “camera on a chip” devices will continue to be enhanced notonly in terms of the quality of imaging which may be achieved, but alsoin the specific construction of the devices which may be manufactured bynew breakthrough processes.

Although the “camera on a chip” concept is one which has great merit forapplication in many industrial areas, a need still exists for a reducedarea imaging device which can be used in even the smallest type ofendoscopic instruments in order to view areas in the body that areparticularly difficult to access, and to further minimize patient traumaby an even smaller diameter invasive instrument.

It is one object of this invention to provide reduced area imagingdevices which take advantage of “camera on a chip” technology, butrearrange the circuitry in a stacked relationship so that there is aminimum profile presented when used within a surgical instrument orother investigative device. It is another object of this invention toprovide low cost imaging devices which may be “disposable.” It is yetanother object of this invention to provide reduced area imaging deviceswhich may be used in conjunction with standard endoscopes by placing theimaging device through channels which normally receive other surgicaldevices, or receive liquids or gases for flushing a surgical area. It isyet another object of this invention to provide a surgical device withimaging capability which may be battery powered and only requires oneconductor for transmitting a pre-video signal to video processingcircuitry within or outside the sterile field of the surgical area.

In addition to the intended use of the foregoing invention with respectto surgical procedures conducted by medical doctors, it is alsocontemplated that the invention described herein has great utility withrespect to oral surgery and general dental procedures wherein a verysmall imaging device can be used to provide an image of particularlydifficult to access locations. Additionally, while the foregoinginvention has application with respect to the medical and dental fields,it will also be appreciated by those skilled in the art that the smallsize of the imaging device set forth herein can be applied to otherfunctional disciplines wherein the imaging device can be used to viewdifficult to access locations for industrial equipment and the like.Therefore, the imaging device of this invention could be used to replacemany industrial boroscopes.

The “camera on a chip” technology can be furthered improved with respectto reducing its profile area and incorporating such a reduced areaimaging device into very small investigative instruments which can beused in the medical, dental, or other industrial fields.

DISCLOSURE OF THE INVENTION

In accordance with the present invention, reduced area imaging devicesare provided. The term “imaging device” as used herein describes theimaging elements and processing circuitry which is used to produce avideo signal which may be accepted by a standard video device such as atelevision or video monitor accompanying a personal computer. The term“image sensor” as used herein describes the components of a solid stateimaging device which captures images and stores them within thestructure of each of the pixels in the array of pixels found in theimaging device. As further discussed below, the timing and controlcircuits can be placed either on the same planar structure as the pixelarray, in which case the image sensor can also be defined as anintegrated circuit, or the timing and control circuitry can be placedremote from the pixel array. The terms “signal” or “image signal” asused herein, and unless otherwise more specifically defined, refer to animage which at some point during its processing by the imaging device,is found in the form of electrons which have been placed in a specificformat or domain. The term “processing circuitry” as used herein refersto the electronic components within the imaging device which receive theimage signal from the image sensor and ultimately place the image signalin a usable format. The terms “timing and control circuits” or“circuitry” as used herein refer to the electronic components whichcontrol the release of the image signal from the pixel array.

In a first embodiment, the image sensor, with or without the timing andcontrol circuitry, may be placed at the distal tip of the endoscopicinstrument while the remaining processing circuitry may be found in asmall remote control box which may communicate with the image sensor bya single cable.

In a second embodiment, the image sensor and the processing circuitrymay all be placed in a stacked arrangement of circuit boards andpositioned at the distal tip of the endoscopic instrument. In thisembodiment, the pixel array of the image sensor may be placed by itselfon its own circuit board while the timing and control circuitry andprocessing circuitry are placed on one or more other circuit boards.Alternatively, the circuitry for timing and control may be placed withthe pixel array on one circuit board, while the remaining processingcircuitry can be placed on one or more of the other circuit boards.

In another embodiment, the imaging device may be adapted for use with astandard rod lens endoscope wherein the imaging device is placed withina standard camera housing which is configured to connect to a standard“C” or “V” mount connector.

In yet another embodiment, the imaging device may be configured so thatthe processing circuitry is placed in the handle of the endoscope, whicheliminates the necessity of having a remote box when the processingcircuitry is remoted from the pixel array. In this embodiment, the pixelarray and the timing and control circuitry are placed at the distal tipof the endoscopic instrument, while the processing circuitry is placedwithin the handle of the endoscope.

A generic endoscope may be used in the each of the embodiments whichincludes a very small diameter tubular portion which is inserted withinthe patient. The tubular portion may be made of a flexible materialhaving a central lumen or opening therein for receiving the elements ofthe imaging device. The tubular portion may be modified to include anadditional concentric tube placed within the central lumen and whichenables a plurality of light fibers to be placed circumferentiallyaround the periphery of the distal end of the tubular portion.Additionally, control wires may extend along the tubular portion inorder to make the endoscope steerable. The material used to make theendoscope can be compatible with any desired sterilization protocol, orthe entire endoscope can be made sterile and disposable after use.

For the configuration of the imaging device which calls for the array ofpixels and the timing and control circuitry to be placed on the samecircuit board, only one conductor is required in order to transmit theimage signal to the processing circuitry. In the other configuration ofthe imaging device wherein the timing and control circuits areincorporated onto other circuit boards, a plurality of connections arerequired in order to connect the timing and control circuitry to thepixel array and the one conductor is also required to transmit the imagesignal.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1a illustrates a first embodiment including a fragmentarycross-sectional view of a generic endoscopic instrument, and afragmentary perspective view of a control box, the endoscope and controlbox each incorporating elements of a reduced area imaging device;

FIG. 1b is an enlarged fragmentary partially exploded perspective viewof the distal end of the endoscopic instrument specifically illustratingthe arrangement of the image sensor with respect to the other elementsof the tubular portion of the endoscope;

FIG. 2a is a fragmentary cross-sectional view of a second embodiment ofthis invention illustrating another generic endoscope wherein theimaging device is incorporated in its entirety at the distal tip of theendoscope;

FIG. 2b is an enlarged fragmentary partially exploded perspective viewof the distal end of the endoscope of FIG. 2a illustrating the imagingdevice;

FIG. 3a is an elevational fragmentary cross-sectional view of the imagesensor incorporated with a standard camera housing for connection to arod lens endoscope;

FIG. 3b is a fragmentary cross-sectional view of the imaging deviceincorporated within the camera housing of FIG. 3a;

FIG. 3c is a fragmentary cross-sectional view similar to that of FIG. 3billustrating a battery as an alternate source of power;

FIG. 4 is a schematic diagram of the functional electronic componentswhich make up the imaging device;

FIG. 4a is an enlarged schematic diagram of a circuit board which mayinclude the array of pixels and the timing and control circuitry;

FIG. 4b is an enlarged schematic diagram of a video processing boardhaving placed thereon the processing circuitry which processes thepre-video signal generated by the array of pixels and which converts thepre-video signal to a post-video signal which may be accepted by astandard video device;

FIGS. 5a-5 e are schematic diagrams that illustrate an example ofspecific circuitry which may be used to make the imaging device.

FIG. 6 is a simplified schematic diagram of a passive pixel which may beplaced in an array of passive pixels compatible with an imager of CMOStype construction;

FIG. 7a illustrates another preferred embodiment including a fragmentarycross-sectional view of a generic endoscope wherein the handle of theendoscope houses processing circuitry of the imaging device;

FIG. 7b is an enlarged fragmentary partially exploded perspective viewof the distal end of the endoscope specifically illustrating thearrangement of the image sensor with respect to the other elements ofthe tubular portion of the endoscope;

FIG. 8a is another fragmentary cross-sectional view of the genericendoscope of FIG. 7a, but showing only one processing circuitry elementwithin the handle of the endoscope; and

FIG. 8b is an enlarged fragmentary partially exploded perspective viewof the distal end of the endoscope of FIG. 8a specifically illustratingthe array of pixels being placed on one planar structure, and the timingand control circuitry being placed on another planar structure adjacentto the pixel array.

BEST MODE FOR CARRYING OUT THE INVENTION

In accordance with one embodiment of the invention as shown in FIG. 1a,an endoscope 10 is provided which incorporates a reduced area imagingdevice 11, shown in FIG. 1b. As further discussed below, the elements ofthe imaging device may all be found at one location or the elements maybe separated from one another and interconnected by the appropriatecable(s). The array of pixels making up the image sensor captures imagesand stores them in the form of electrical energy by conversion of lightphotons to electrons. This conversion takes place by the photo diodes ineach pixel which communicate with one or more capacitors which store theelectrons. The structure of the endoscope 10 includes a flexible orrigid tubular portion 14 which is inserted into the body of the patientand is placed at the appropriate location for viewing a desired surgicalarea. The tubular portion 14 attaches at its proximal end to a handleportion 12 which may be grasped by a surgeon who is conducting theendoscopic procedure. The handle 12 may include a central lumen orchannel 13 which receives therethrough one or more cables or otherstructures which extend to the distal end 16 of tubular portion 14.Handle portion 12 may further include a supplementary channel 15 whichintersects with central channel 13 and which may provide another pointof entry for other cables, fluids or operative instruments to be placedthrough the endoscope.

FIG. 1b illustrates the distal end of the endoscope 16. The distal end16 may be characterized by an outer tube 18 which traverses the lengthof the tubular portion 14 and connects to the handle portion 12. Placedconcentrically within the outer tube 18 may be one or more inner tubes20. In FIG. 1b, the gap between inner tube 20 and outer tube 18 forms aspace in which one or more light fibers 22 or control wires 24 may beplaced. As well understood by those skilled in the art, a plurality ofcircumferentially spaced light fibers as illustrated in FIG. 1b can beused to illuminate the surgical site. Additionally, the control wires 24may communicate with a control mechanism (not shown) integrated on thehandle portion 12 for manipulating the distal end 16 of the endoscope ina desired direction. The flexible tubular portion 14 coupled with asteerable feature enables the endoscope to be placed within windingbodily passages or other locations difficult to reach within the body.

An image sensor 40 may be placed within the central channel defined byinner tube 20. In the configuration shown in FIG. 1b, a cable 26 is usedto house the conductors which communicate with the image sensor 40. Anintermediate support tube 28 may be placed concentrically outside ofcable 26 and concentrically within inner tube 20 to provide thenecessary support for the cable 26 as it traverses through the innerchannel defined by inner tube 20. In lieu of support tube 28, otherwell-known means may be provided to stabilize the cable 26 such as clipsor other fastening means which may attach to the inner concentricsurface of inner tube 20.

A control box 30 may be placed remote from the endoscope 10. The controlbox 30 contains some of the processing circuitry which is used toprocess the image signal produced by image sensor 40. Therefore, theimaging device 11 as previously defined would include the processingcircuitry within control box 30 and the image sensor 40 located at thedistal tip of the endoscope. Control box 30 communicates with imagesensor 40 by means of cable 32 which may simply be an insulated andshielded cable which houses therein cable 26. Cable 32 is stabilizedwith respect to the handle portion 12 by means of a fitting 34 whichensures that cable 32 cannot be inadvertently pushed or pulled withinchannel 13. Additionally, an additional fitting 35 may be provided tostabilize the entry of a light cable 36 which houses the plurality oflight fibers 22.

Image sensor 40 is illustrated as being a planar and square shapedmember. However, the image sensor may be modified to be in a planar andcircular shape to better fit within the channel defined by inner tube20. Accordingly, FIG. 1b further shows an alternate shaped image sensor40′ which is round. A lens group or system 42 may be incorporated at thedistal end of the endoscope in order to manipulate the image prior to itbeing impinged upon the array of pixels on the image sensor 40. Thislens system 42 may be sealed at the distal end 16 of the endoscope sothat the tubular portion 14 is impervious to fluids entering through thedistal end 16. In the configuration of the imaging device 11 in FIGS. 1aand 1 b, there are only three conductors which are necessary forproviding power to the image sensor 40, and for transmitting an imagefrom the image sensor 40 back to the processing circuitry found withincontrol box 30. Namely, there is a power conductor 44, a groundingconductor 46, and an image signal conductor 48 each of which are hardwired to the image sensor. Thus, cable 26 may simply be athree-conductor 50 ohm cable.

Image sensor 40 can be as small as 1 mm in its largest dimension.However, a more preferable size for most endoscopic procedures woulddictate that the image sensor 40 be between 4 mm to 8 mm in its largestdimension. The image signal transmitted from the image sensor throughconductor 48 is also herein referred to as a pre-video signal. Once thepre-video signal has been transmitted from image sensor 40 by means ofconductor 48, it is received by video processing board 50. Videoprocessing board 50 then carries out all the necessary conditioning ofthe pre-video signal and places it in a form so that it may be vieweddirectly on a standard video device, television or standard computervideo monitor. The signal produced by the video processing board 50 canbe further defined as a post-video signal which can be accepted by astandard video device. As shown in FIG. 1a, a conductor 49 is providedwhich transmits the post-video signal to an output connector 58 on theexterior surface of control box 30. The cable (not shown) extending fromthe desired video device (not shown) may receive the post-video signalby means of connector 58. Power supply board 52 may convert incomingpower received through power source 54 into the desired voltage. In thepreferred imager incorporated in this invention, the power to theimaging device is simply a direct current which can be a 1.5 volt to a12 volt source. Incoming power from, for example, a wall receptacle,communicates with power supply board 52 by connector 56. Power supplyboard 52 takes the incoming power source and regulates it to the desiredlevel. Additionally, ground 46 is also shown as extending back to thesource of power through connector 56.

FIG. 2a illustrates a second embodiment of this invention wherein theimaging device is self-contained entirely within the distal end 16 ofthe endoscope, and a power source which drives the circuitry within theimaging device may come from a battery 66 housed within handle portion12.

As shown in FIG. 2b, the video processing board 50 may be placeddirectly behind image sensor 40. A plurality of pin connectors 62 serveto electrically couple image sensor 40 with video processing board 50depending upon the specific configuration of image sensor 40, pinconnectors 62 may be provided either for structural support only, or toprovide a means by which image signals are transmitted between imagesensor 40 and board 50. When necessary, one or more supplementary boards60 may be provided which further contain processing circuitry to processthe image signal and present it in a form which may be directly receivedby a desired video device. The area which is occupied by image sensor 40may be defined as the profile area of the imaging device and whichdetermines its critical dimensions. Any imaging elements that are foundon boards 50 or 60 must be able to be placed on one or more circuitboards which are longitudinally aligned with image sensor 40 alonglongitudinal axis XX. If the profile area is not critical in terms oflimiting the largest sized imaging element within the imaging device,then the additional circuit boards 50 and 60 which are normally placedin line with image sensor 40 can be aligned in an offset manner or maybe larger than the profile area of image sensor 40. In the configurationof FIG. 2b, it is desirable that elements 40, 50 and 60 be approximatelythe same size so that they may fit uniformly within the central channelof the endoscope. Additionally, image sensor 40 may be bonded to lenssystem 42 in order to provide further structural support to the imagingdevice 11 when mounted within the distal end 16.

Referring back to the handle portion 12 in FIG. 2a, an additionalchannel 64 may be provided in order that a power supply cable 68 maycommunicate with battery 66. Conveniently, battery 66 may itself bemounted within a well 65 formed in handle portion 12. Cable 68 carriesthe conductor 44 and ground 46. Cable 68 may intersect with cable 33within channel 13, cables 68 and 33 extending then to the distal end 16.Cable 33 can be a single conductor cable which transmits the post-videosignal to a desired video device. In other words, cable 33 may simply bean insulated and shielded housing for conductor 49 which carries thepost-video signal. Because a preferred image sensor of the imagingdevice 11 may only require a 5 volt power supply, a battery is an idealpower source in lieu of a conductor which would trail the endoscope.Accordingly, the endoscope is made more mobile and easier to handle byeliminating at least one of the trailing cables.

FIG. 3a illustrates yet another preferred embodiment of this invention,wherein the imaging device can be used in conjunction with a standardrod lens endoscope 70. As shown, rod lens endoscope 70 includes a lenstrain 72 which includes a plurality of highly precise lenses (not shown)which are able to transmit an image from the distal end of theendoscope, to a camera in line with the endoscope. The rod lensendoscope is equipped with a light guide coupling post 74. Light guidepost 74 connects to a source of light in the form of a cable 77 having aplurality of fiber optic strands (not shown) which communicate with asource of light (not shown). The most common arrangement of the rod lensendoscope also includes a “C” or “V” mount connector 78 which attachesto the eyepiece 76. The “C” or “V” mount attaches at its other end to acamera group 80. The camera group 80 houses one or more of the elementsof the imaging device. In this embodiment, the small size of the imagingdevice is not a critical concern since the imaging device is not beingplaced at the distal end of the endoscope. However, the incorporation ofthe imaging device in a housing which would normally hold a traditionalcamera still provides an advantageous arrangement. As shown, the cameragroup 80 may include a housing 82 which connects to a power/video cable86. Fitting 87 is provided to couple cable 86 to the interior elementsof the camera group 80 found within housing 82. FIG. 3a illustrates anarrangement of the imaging device 11 wherein the image sensor 40 isplaced by itself within the housing 82 and the processing circuitry ofthe imaging device can be positioned in a remote control box as shown inFIG. 1a. Accordingly, only three conductors 44, 46 and 48 are necessaryfor providing power to the image sensor 40 and for transmitting thepre-video signal to the control box. Alternatively, as shown in FIG. 3b,the entire imaging device 11 may be incorporated within camera group 80,each of the elements of the imaging device being placed in the stackedarrangement similar to FIG. 2b. As discussed above, size is not as muchof a concern in the embodiment of FIGS. 3a and 3 b since the cameragroup housing 82 is much larger than the distal tip of the endoscope ofFIGS. 1a and 2 a.

FIG. 3c also illustrates the use of a battery 66 which provides sourceof power to the imaging device in either FIGS. 3a or 3 b. In thisarrangement, housing 82 is altered to include a battery housing 69 whichhouses the battery 66 therein. Battery housing 69 may include a verysmall diameter channel which may allow conductor 48 or 49 to communicatedirectly with the processing circuitry or video device, respectively. Itwill also be understood that the embodiment in FIG. 1a may incorporatethe use of a battery 66 as the source of power. Thus, handle 12 in FIG.1a may be altered in the same way as housing 82 to allow a battery to beattached to the handle portion 12.

FIG. 4 is a schematic diagram illustrating one way in which the imagingdevice 11 may be constructed. As illustrated, the image sensor 40 mayinclude the timing and control circuits on the same planar structure.Power is supplied to image sensor 40 by power supply board 52. Theconnection between image sensor 40 and board 52 may simply be a cablehaving two conductors therein, one for ground and another fortransmitting the desired voltage. These are illustrated as conductors 44and 46. The output from image sensor 40 in the form of the pre-videosignal is input to video processor board 50 by means of the conductor48. In the configuration of FIG. 4, conductor 48 may simply be a 50 ohmconductor. Power and ground also are supplied to video processing board50 by conductors 44 and 46 from power supply board 52. The output signalfrom the video processor board 50 is in the form of the post-videosignal and which may be carried by conductor 49 which can also be a 50ohm conductor.

In the first embodiment illustrated in FIG. 1a, cable 32 can be used tohouse conductors 44, 46 and 48. In the embodiment shown in FIG. 2a,cable 33 can be used to house conductor 49 by itself when a batterypower source is used, or alternatively, cable 33 may house conductors44, 46 and 49 if the embodiment of FIG. 2a utilizes a power source fromboard 52.

Optionally, a supplementary processing board 60 may be provided tofurther enhance the pre-video signal. As shown in FIG. 4, thesupplementary board 60 may be placed such that the pre-video signal fromimage sensor 40 is first sent to the supplementary board and then outputto the video processor board 50. In this case, the output from board 50can be carried along conductor 51. This output can be defined as anenhanced pre-video signal. Furthermore, the post-video signal from videoprocessor board 50 may return to the supplementary board 60 for furtherprocessing, as further discussed below. The conductor used to transmitthe post-video signal back to the supplementary board is shown asconductor 59. The power supply board 52 may also provide power to thesupplementary board in the same manner as to image sensor 40 and board50. That is, a simple hard-wired connection is made onto thesupplementary board for the ground and voltage carrying conductors. Asdiscussed above, image sensor 40 may be placed remotely from boards 50and 60. Alternatively, image sensor 40, and boards 50 and 60 each may beplaced within the distal end of the endoscope.

Although FIG. 4 illustrates the image sensor and the timing and controlcircuits being placed on the same planar structure, it is possible toseparate the timing and control circuits from the pixel array and placethe timing and control circuits onto video processing board 50. Theadvantage in placing the timing and control circuits on the same planarstructure as the image sensor is that only three connections arerequired between image sensor 40 and the rest of the imaging device,namely, conductors 44, 46 and 48. Additionally, placing the timing andcontrol circuits on the same planar structure with the pixel arrayresults in the pre-video signal having less noise. Furthermore, theaddition of the timing and control circuits to the same planar structurecarrying the image sensor only adds a negligible amount of size to onedimension of the planar structure. If the pixel array is to be the onlyelement on the planar structure, then additional connections must bemade between the planar structure and the video processing board 50 inorder to transmit the clock signals and other control signals to thepixel array. For example, a ribbon-type cable (not shown) or a pluralityof 50 ohm coaxial cables (not shown) must be used in order to controlthe downloading of information from the pixel array. Each of theseadditional connections would be hard wired between the boards.

FIG. 4a is a more detailed schematic diagram of image sensor 40 whichcontains an array of pixels 90 and the timing and control circuits 92.One example of a pixel array 90 which can be used within the inventionis similar to that which is disclosed in U.S. Pat. No. 5,471,515 toFossum, et al., said patent being incorporated by reference herein. Morespecifically, FIG. 3 of Fossum, et al. illustrates the circuitry whichmakes up each pixel in the array of pixels 90. The array of pixels 90 asdescribed in Fossum, et al. is an active pixel group with intra-pixelcharged transfer. The image sensor made by the array of pixels is formedas a monolithic complementary metal oxide semiconductor integratedcircuit which may be manufactured in an industry standard complementarymetal oxide semiconductor process. The integrated circuit includes afocal plane array of pixel cells, each one of the cells including aphoto gate overlying the substrate for accumulating the photo generatedcharges. In broader terms, as well understood by those skilled in theart, an image impinges upon the array of pixels, the image being in theform of photons which strike the photo diodes in the array of pixels.The photo diodes or photo detectors convert the photons into electricalenergy or electrons which are stored in capacitors found in each pixelcircuit. Each pixel circuit has its own amplifier which is controlled bythe timing and control circuitry discussed below. The information orelectrons stored in the capacitors is unloaded in the desired sequenceand at a desired frequency, and then sent to the video processing board50 for further processing.

Although the active pixel array disclosed in U.S. Pat. No. 5,471,515 ismentioned herein, it will be understood that the hybrid CCD/CMOSdescribed above, or any other solid state imaging device may be usedwherein timing and control circuits can be placed either on the sameplanar structure with the pixel array, or may be separated and placedremotely. Furthermore, it will be clearly understood that the inventionclaimed herein is not specifically limited to an image sensor asdisclosed in the U.S. Pat. No. 5,471,515, but encompasses any imagesensor which may be configured for use in conjunction with the otherprocessing circuitry which makes up the imaging device of thisinvention.

The timing and control circuits 92 are used to control the release ofthe image information or image signal stored in the pixel array. In theimage sensor of Fossum, et al., the pixels are arranged in a pluralityof rows and columns. The image information from each of the pixels isfirst consolidated in a row by row fashion, and is then downloaded fromone or more columns which contain the consolidated information from therows. As shown in FIG. 4a, the control of information consolidated fromthe rows is achieved by latches 94, counter 96, and decoder 98. Theoperation of the latches, counter and decoder is similar to theoperation of similar control circuitry found in other imaging devices.That is, a latch is a means of controlling the flow of electrons fromeach individual addressed pixel in the array of pixels. When a latch 94is enabled, it will allow the transfer of electrons to the decoder 98.The counter 96 is programmed to count a discrete amount of informationbased upon a clock input from the timing and control circuits 92. Whenthe counter 96 has reached its set point or overflows, the imageinformation is allowed to pass through the latches 94 and be sent to thedecoder 98 which places the consolidated information in a serial format.Once the decoder 98 has decoded the information and placed it in theserial format, then the row driver 100 accounts for the serialinformation from each row and enables each row to be downloaded by thecolumn or columns. In short, the latches 94 will initially allow theinformation stored in each pixel to be accessed. The counter 96 thencontrols the amount of information flow based upon a desired timesequence. Once the counter has reached its set point, the decoder 98then knows to take the information and place it in the serial format.The whole process is repeated, based upon the timing sequence that isprogrammed. When the row driver 100 has accounted for each of the rows,the row driver reads out each of the rows at the desired video rate.

The information released from the column or columns is also controlledby a series of latches 102, a counter 104 and a decoder 106. As with theinformation from the rows, the column information is also placed in aserial format which may then be sent to the video processing board 50.This serial format of column information is the pre-video signal carriedby conductor 48. The column signal conditioner 108 places the columnserial information in a manageable format in the form of desired voltagelevels. In other words, the column signal conditioner 108 only acceptsdesired voltages from the downloaded column(s).

The clock input to the timing and control circuits 92 may simply be aquartz crystal timer. This clock input is divided into many otherfrequencies for use by the various counters. The run input to the timingand control circuit 92 may simply be an on/off control. The defaultinput can allow one to input the pre-video signal to a video processorboard which may run at a frequency of other than 30 hertz. The datainput controls functions such as zoom. At least for a CMOS type activepixel array which can be accessed in a random manner, features such aszoom are easily manipulated by addressing only those pixels which locatea desired area of interest by the surgeon.

A further discussion of the timing and control circuitry which may beused in conjunction with an active pixel array is disclosed in U.S. Pat.No. 5,471,515 and is also described in an article entitled “Active PixelImage Sensor Integrated With Readout Circuits” appearing in NASA TechBriefs, October 1996, pp. 38 and 39. This particular article is alsoincorporated by reference.

Once image sensor 40 has created the pre-video signal, it is sent to thevideo processing board 50 for further processing. At board 50, as shownin FIG. 4b, the pre-video signal is passed through a series of filters.One common filter arrangement may include two low pass filters 114 and116, and a band pass filter 112. The band pass filter only passes lowfrequency components of the signal. Once these low frequency componentspass, they are then sent to detector 120 and white balance circuit 124,the white balance circuit distinguishing between the colors of red andblue. The white balance circuit helps the imaging device set its normal,which is white. The portion of the signal passing through low passfilter 114 then travels through gain control 118 which reduces themagnitude or amplitude of this portion to a manageable level. The outputfrom gain control 118 is then fed back to the white balance circuit 124.The portion of the signal traveling through filter 116 is placed throughthe processor 122. In the processor 122, the portion of the signalcarrying the luminance or non-chroma is separated and sent to the Ychroma mixer 132. Any chroma portion of the signal is held in processor122.

Referring to the output of the white balance circuit 124, this chromaportion of the signal is sent to a delay line 126 where the signal isthen further reduced by switch 128. The output of switch 128 is sentthrough a balanced modulator 130 and also to the Y chroma mixer 132where the processed chroma portion of the signal is mixed with theprocessed non-chroma portion. Finally, the output from the Y chromamixer 132 is sent to the NTSC/PAL encoder 134, commonly known in the artas a “composite” encoder. The composite frequencies are added to thesignal leaving the Y chroma mixer 132 in encoder 134 to produce thepost-video signal which may be accepted by a television.

Referring back to FIG. 4, it further illustrates supplementary board 60which may be used to digitally enhance or otherwise further conditionthe pre-video signal produced from image sensor 40. For example, digitalenhancement can brighten or otherwise clarify the edges of an imageviewed on a video screen. Additionally, the background images may beremoved thus leaving only the foreground images or vice versa. Theconnection between image sensor 40 and board 60 may simply be theconductor 48 which may also transfer the pre-video signal to board 50.Once the pre-video signal has been digitally enhanced on supplementaryboard 60, it is then sent to the video processor board 50 by means ofanother conductor 51. The pre-video signal is an analog signal. Thedigitally enhanced pre-video signal may either be a digital signal or itmay be converted back to the analog domain prior to being sent to board50.

In addition to digital enhancement, supplementary board 60 may furtherinclude other circuitry which may further condition the post-videosignal so that it may be viewed in a desired format other than NTSC/PAL.As shown in FIGS. 4, intermediate conductor 59 may transmit the signaloutput from Y chroma mixer 132 back to the supplementary board 60 wherethe signal is further encoded for viewing in a particular format. Onecommon encoder which can be used includes an RGB encoder 154. The RGBencoder separates the signal into three separate colors (red, green andblue) so that the surgeon may selectively choose to view only thoseimages containing one or more of the colors. Particularly in tissueanalysis where dyes are used to color the tissue, the RGB encoder mayhelp the surgeon to identify targeted tissue.

The next encoder illustrated in FIG. 4 is a SVHS encoder 156 (supervideo home system). This encoder splits or separates the luminanceportion of the signal and the chroma portion of the signal prior toentering the video device. Some observers believe that a cleaner signalis input to the video device by such a separation which in turn resultsin a more clear video image viewed on the video device. The last encoderillustrated in FIG. 4 is a VGA encoder 158 which enables the signal tobe viewed on a standard VGA monitor which is common to many computermonitors.

One difference between the arrangement of image sensor 40 and theoutputs found in FIG. 3 of the Fossum, et al. patent is that in lieu ofproviding two analog outputs [namely, VS out (signal) and VR out(reset)], the reset function takes place in the timing and controlcircuitry 92. Accordingly, the pre-video signal only requires oneconductor 48.

FIGS. 5a-5 e illustrate in more detail one example of circuitry whichmay be used in the video processing board 50 in order to produce apost-video signal which may be directly accepted by a video device suchas a television. The circuitry disclosed in FIGS. 5a-5 e is very similarto circuitry which is found in a miniature quarter-inch Panasoniccamera, Model KS-162. It will be understood by those skilled in the artthat the particular arrangement of elements found in FIGS. 5a-5 e areonly exemplary of the type of video processing circuitry which may beincorporated in order to take the pre-video signal and condition it tobe received by a desired video device.

As shown in FIG. 5a, 5 volt power is provided along with a ground byconductors 44 and 46 to board 50. The pre-video signal carried byconductor 48 is buffered at buffer 137 and then is transferred toamplifying group 138. Amplifying group 138 amplifies the signal to ausable level as well as achieving impedance matching for the remainingcircuitry.

The next major element is the automatic gain control 140 shown in FIG.5b. Automatic gain control 140 automatically controls the signal fromamplifying group 138 to an acceptable level and also adds othercharacteristics to the signal as discussed below. More specifically,automatic gain control 140 conditions the signal based upon inputs froma 12 channel digital to analog converter 141. Converter 141 retrievesstored information from EEPROM (electrically erasable programmable readonly memory) 143. EEPROM 143 is a non-volatile memory element which maystore user information, for example, settings for color, tint, balanceand the like. Thus, automatic gain control 140 changes the texture orvisual characteristics based upon user inputs. The signal leaving theautomatic gain control 140 is an analog signal until being converted byanalog to digital converter 142.

Digital signal processor 144 of FIG. 5c further processes the convertedsignal into a serial type digital signal. One function of themicroprocessor 146 is to control the manner in which digital signalprocessor 144 sorts the digital signals emanating from converter 142.Microprocessor 146 also controls analog to digital converter 142 interms of when it is activated, when it accepts data, when to releasedata, and the rate at which data should be released. Microprocessor 146may also control other functions of the imaging device such as whitebalance. The microprocessor 146 may selectively receive the informationstored in the EEPROM 143 and carry out its various commands to furthercontrol the other elements within the circuitry.

After the signal is processed by digital signal processor 144, thesignal is sent to digital encoder 148 illustrated in FIG. 5d. Some ofthe more important functions of digital encoder 148 are to encode thedigital signal with synchronization, modulated chroma, blanking,horizontal drive, and the other components necessary so that the signalmay be placed in a condition for reception by a video device such as atelevision monitor. As also illustrated in FIG. 5d, once the signal haspassed through digital encoder 148, the signal is reconverted into ananalog signal through digital to analog converter 150.

This reconverted analog signal is then buffered at buffers 151 and thensent to amplifier group 152 of FIG. 5e which amplifies the signal sothat it is readily accepted by a desired video device. Specifically, asshown in FIG. 5e, one SVHS outlet is provided at 160, and two compositeor NTSC outlets are provided at 162 and 164, respectively.

In addition to the active pixel-type CMOS imager discussed above,certain advances in passive pixel-type CMOS imagers have been made suchthat the traditional noise associated with such passive arrangements canbe overcome by improved manufacturing technologies which therefore doesnot require each signal to be amplified at each pixel site. Accordingly,FIG. 6 illustrates a simplified schematic diagram of a passive pixelwhich may be incorporated directly into the read out circuitry ofFossum, et al. (see FIG. 3, U.S. Pat. No. 5,471,515; read out circuit orcorrelated double sampling circuit 70). As shown in FIG. 6, each passivepixel 160 in a passive pixel array comprises a photo diode 162 with atransistor 164 that passes the photoelectrically generated signal fromphoto diode 162 to a charge integration amplifier (not shown) outsidethe pixel array. After photo charge integration, the timing and controlcircuitry activates the access transistor 164. The photoelectricallygenerated signal from photo diode 162 then transfers to the capacitanceof the column bus 166 where the charge integration amplifier (not shown)at the end of the column bus 166 senses the resulting voltage. Thecolumn bus voltage resets the photo diode 162, and the timing andcontrol circuitry then places the access transistor 164 in an offcondition. The pixel 160 is then ready for another integration cycle.The signal output from either the active or passive pixel arrays areprocessed identically. Accordingly, FIG. 6 illustrates that the readoutcircuit 70 of Fossum, et al. is compatible with either the active orpassive pixel arrays disclosed herein. One example of a manufacturer whohas developed a passive pixel array with performance nearly equal tothat of known active pixel devices and compatible with the read outcircuitry of Fossum, et al. is VLSI Vision Ltd., 1190 Saratoga Avenue,Suite 180, San Jose, Calif. 95129.

FIGS. 7a and 7 b illustrate yet another preferred embodiment of thisinvention. This embodiment also incorporates a generic endoscope, suchas shown in FIGS. 1a and 2 a. Specifically, the generic endoscope 170includes a handle 172 which may be grasped by the surgeon. The handle172 has an interior opening 173 which allows wiring to pass through tothe distal tip 177 of the endoscope. This interior opening 173, asfurther discussed below, also houses the processing circuitry of theimaging device. The generic endoscope further includes a tubular portion174 which is placed within the patient's body and which is defined by aflexible outer tube 178. A battery channel 175 may also be incorporatedwithin the handle 172 to receive a battery 176. FIG. 7b shows the distaltip 177 of the endoscope in an enlarged fashion. A lens system 180 maybe used to manipulate an image. Images are received upon a planarstructure in the form of an image sensor 182 which includes an array ofpixels and corresponding timing and control circuitry. This planarstructure is the same as that illustrated in FIG. 4a. Image sensor 182incorporating the pixel array and timing and control circuitry producesa pre-video signal (either analog or digital) which is transmitted bypre-video out conductor 188. A 5-volt power source and a ground areprovided to image sensor 182 by conductors 184 and 186, respectively. Aprotective cable or sheathing 190 houses conductors 184, 186 and 188 asthey extend proximally back toward the handle 172 of the endoscope 170.Additionally, a support tube 192 may fit over the protective cable 190to provide further protection for the conductors. Referring back to FIG.7a, desired processing circuitry can be placed directly within thehandle of the endoscope since the processing circuitry is such a smallsize. In FIG. 7a, the processing circuitry incorporated within thehandle 172 includes two planar structures, namely, a supplementary board194 and a video processor board 196. In terms of the construction ofthese boards, the boards 194 and 196 are the same as video processorboard 50 and supplementary board 60, respectively, of the firstembodiment. Boards 194 and 196 may also be spaced apart from one anotherand placed in an aligned position as by pin connectors 195. Pinconnectors 195 are also of the same type as pin connectors 62 shown inFIG. 2b. The pre-video signal transmitted by conductor 188 is processedby the processing circuitry within the handle, and a post-video outsignal is produced and transmitted by post-video out conductor 198.Conductor 198 then connects directly to the desired video device (notshown) such as a video screen or personal computer. As shown in FIG. 7a,5-volt power conductor 184, ground conductor 186, and post-video outconductor 198 may be housed within cable 199 which connects to the videodevice and a source of power (not shown). A fitting 200 may be used tostabilize cable 199 in its attachment to the handle 172. As also shownin FIG. 7a, a light fiber bundle 202 may extend through the endoscope toprovide light to the distal tip 177. Accordingly, a cable 203 wouldextend back to a source of light (not shown), and fitting 204 would beused to stabilize the connection of cable 203 to the handle 172. FIG. 7afurther illustrates a power and ground conductor 206 which extends fromthe battery compartment/channel 175 in order to provide an alternatesource of power to the endoscope. FIG. 7a has been simplified to betterillustrate the differences between it and the previous embodiments.Accordingly, the light fibers and control wires which may extend to thedistal end 177 are not illustrated (corresponding to light fibers 22 andcontrol wires 24 of the first embodiment).

FIGS. 8a and 8 b illustrate another endoscope which differs from FIGS.7a and 7 b by modifications made to the arrangement of the imagingdevice. FIG. 8a also does not illustrate the use of an alternate powersource; however, it shall be understood, of course, that this Figurecould also utilize a battery source of power as shown in FIG. 7a. Morespecifically, FIGS. 8a and 8 b illustrate an imaging device wherein thearray of pixels 208 and the timing and control circuitry 210 are on twoseparate planar structures placed back to back to one another in analigned fashion. A multistrand conductor 212 transmits image signalsproduced by the pixel array 208, and also carries the timing and controlsignals to the pixel array allowing the image signals to be read orunloaded at the desired speed, frequency, and sequence. Also FIG. 8aillustrates the use of video processor board 196, and no supplementaryboard 194. It shall be understood that, for both FIGS. 7a and 8 a, thespecific processing circuitry found within the interior opening 173 ofthe handle can include whatever type of processing circuitry as neededto create a post-video out signal which is readily acceptable by a videodevice without any further processing. Thus, FIG. 7a could be usedwithout supplementary board 194, and FIG. 8a could incorporate the useof supplementary board 194. It shall also be understood that boards 194and 196 have been greatly enlarged to better show their spatialarrangement and detail within interior opening 173. Although it ispossible that these boards may be of such illustrated size as mentionedabove with respect to the previous embodiment and boards 50 and 60,these boards can be made small enough that the opening 173 within theendoscope has ample room to house the processing circuitry therein. Interms of the actual structure which is used to support the processingcircuitry within the handle, the handle may be equipped with anysuitable non-conductive support flanges or other extensions within theinterior opening 173 which would allow the processing circuitry to bemounted thereon. Because of the extremely small size and insignificantweight of the processing circuitry, such supporting structure withininterior opening 173 would be minimal.

From the foregoing, it is apparent that an entire imaging device may beincorporated within the distal tip of an endoscope, or may have someelements of the imaging device being placed in a small remote boxadjacent to the endoscope. Based upon the type of image sensor used, theprofile area of the imaging device may be made small enough to be placedinto an endoscope which has a very small diameter tube. Additionally,the imaging device may be placed into the channels of existingendoscopes to provide additional imaging capability without increasingthe size of the endoscope. The imaging device may be powered by astandard power input connection in the form of a power cord, or a smalllithium battery may be used.

This invention has been described in detail with reference to particularembodiments thereof, but it will be understood that various othermodifications can be effected within the spirit and scope of thisinvention.

What is claimed is:
 1. An endoscope with integral imaging capabilitycomprising: a handle for grasping by a surgeon, said handle having aninterior opening; a tubular portion including a distal end, a proximalend and a central passageway extending therethrough, said tubularportion connected at said proximal end to said handle; an image sensorlying in a first plane and including an array of CMOS pixels, said imagesensor positioned near said distal end of said tubular portion forreceiving images of a surgical site, said image sensor producing animage signal; circuitry means electrically coupled to said image sensorfor timing and control of said image sensor, said circuitry means fortiming and control placed within said tubular portion; a video processorboard placed within said interior opening of said handle and in electriccommunication with said image sensor, said video processor boardincluding circuitry means for processing said image signal; and a powersupply electrically coupled to said image sensor and said videoprocessor board.
 2. An endoscope, as claimed in claim 1, wherein:individual CMOS pixels within said array of CMOS pixels each includes anamplifier.
 3. An endoscope, as claimed in claim 1, wherein: said arrayof CMOS pixels includes a plurality of passive CMOS pixels.
 4. Anendoscope, as claimed in claim 1, wherein: said array of CMOS pixelsincludes a plurality of passive CMOS pixels, and wherein individualpassive CMOS pixels of said plurality of passive CMOS pixels eachincludes a photo diode for producing photoelectrically generatedsignals, and an access transistor communicating with said photo diode tocontrol the release of said photoelectrically generated signals.
 5. Anendoscope, as claimed in claim 1, wherein: said circuitry means fortiming and control is placed on said first plane adjacent said imagesensor.
 6. An endoscope, as claimed in claim 1, wherein: said circuitrymeans for timing and control is placed on a second plane spaced from andadjacent to said first plane near said distal end of said tubularportion.
 7. An endoscope, as claimed in claim 1, further including: alens positioned at said distal end of said tubular portion and distallyof said image sensor for producing a modified image on said imagesensor.
 8. An endoscope, as claimed in claim 1, wherein: said powersupply is a battery attached to said handle.
 9. An endoscope, as claimedin claim 6, wherein: said second plane is offset from and substantiallyparallel to said first plane.
 10. An endoscope, as claimed in claim 1,further including: means for providing light to said distal end of saidtubular portion secured within said tubular portion.
 11. An endoscope,as claimed in claim 1, further including: a supplementary circuit boardfor digitally enhancing said image signal, said supplementary circuitboard being electrically coupled to said video processor board.
 12. Anendoscope, as claimed in claim 1, wherein: said image signal is carriedby a single conductor to said video processor board.